Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(auto-salvage): v2 support #675

Merged
merged 2 commits into from
Nov 6, 2024

Conversation

c3y1huang
Copy link
Contributor

Which issue(s) this PR fixes:

Issue longhorn/longhorn#8430

What this PR does / why we need it:

Include snapshot disk info in the proxy.ReplicaList response.

Special notes for your reviewer:

None

Additional documentation or context

None

@c3y1huang c3y1huang self-assigned this Sep 19, 2024
Copy link

mergify bot commented Sep 21, 2024

This pull request is now in conflict. Could you fix it @c3y1huang? 🙏

@c3y1huang c3y1huang force-pushed the 8430-feat-v2-auto-salvage branch 4 times, most recently from 1f4b165 to 5eacb5c Compare September 27, 2024 02:12
go.mod Outdated Show resolved Hide resolved
Copy link

mergify bot commented Oct 9, 2024

This pull request is now in conflict. Could you fix it @c3y1huang? 🙏

Copy link

coderabbitai bot commented Oct 24, 2024

Walkthrough

The changes introduce a new boolean field, SalvageRequested, to the EngineCreateRequest struct in pkg/client/instance.go, allowing indication of salvage operations during engine creation. The InstanceCreate method in V2DataEngineInstanceOps is updated to pass this new field to the engine creation process. Additionally, the InstanceDelete method is modified to improve handling of cleanup requirements based on instance type. Overall, these changes enhance the instance management functionality in the V2 data engine.

Changes

File Change Summary
pkg/client/instance.go Added SalvageRequested bool to EngineCreateRequest struct; updated InstanceCreate method to use this field.
pkg/instance/instance.go Updated InstanceCreate to include SalvageRequested in parameters; modified InstanceDelete for cleanup logic; no changes to InstanceGet.

Assessment against linked issues

Objective Addressed Explanation
Support for auto salvage in v2 volume (#8430)

📜 Recent review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between b3f1700 and f324a73.

⛔ Files ignored due to path filters (7)
  • go.mod is excluded by !go.mod
  • go.sum is excluded by !**/*.sum, !go.sum
  • vendor/github.com/longhorn/longhorn-spdk-engine/pkg/client/client.go is excluded by !vendor/**
  • vendor/github.com/longhorn/longhorn-spdk-engine/pkg/spdk/engine.go is excluded by !vendor/**
  • vendor/github.com/longhorn/longhorn-spdk-engine/pkg/spdk/replica.go is excluded by !vendor/**
  • vendor/github.com/longhorn/longhorn-spdk-engine/pkg/spdk/server.go is excluded by !vendor/**
  • vendor/modules.txt is excluded by !vendor/**
📒 Files selected for processing (2)
  • pkg/client/instance.go (2 hunks)
  • pkg/instance/instance.go (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
  • pkg/client/instance.go
  • pkg/instance/instance.go

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Outside diff range and nitpick comments (7)
pkg/proxy/volume.go (1)

74-77: Document the purpose of the boolean parameter in EngineGet.

The addition of false parameter to c.EngineGet() needs documentation to explain its purpose and impact, especially since it appears to be related to the auto salvage feature.

Consider adding a comment explaining what this boolean parameter controls:

+       // The boolean parameter controls whether to include salvage-related information
        recv, err := c.EngineGet(req.EngineName, false)
pkg/proxy/replica.go (2)

154-155: Document the purpose of the boolean parameter in EngineGet.

The false parameter's purpose is not immediately clear from the context. Please add a comment explaining what this boolean parameter controls and why it's set to false here.


231-232: Maintain consistent error handling patterns.

The error handling here differs from the ReplicaList method. Consider using errors.Wrapf for consistent error handling:

-return nil, grpcstatus.Errorf(grpccodes.Internal, "failed to get engine %v: %v", req.EngineName, err)
+return nil, grpcstatus.Error(grpccodes.Internal, errors.Wrapf(err, "failed to get engine %v", req.EngineName).Error())
pkg/proxy/snapshot.go (4)

Line range hint 158-165: Copy snapshot labels into the response.

Currently, the Labels field in resp.Disks is initialized as an empty map. If the snapshot objects contain labels, consider copying them to preserve important metadata.

Apply this diff to include the labels:

resp.Disks[snapshotName] = &rpc.EngineSnapshotDiskInfo{
	Name:        snapshot.Name,
	Parent:      snapshot.Parent,
	Children:    snapshot.Children,
	Removed:     false,
	UserCreated: snapshot.UserCreated,
	Created:     snapshotTime,
	Size:        strconv.FormatUint(snapshot.ActualSize, 10),
-	Labels:      map[string]string{},
+	Labels:      snapshot.Labels,
}

Line range hint 271-273: Correct the error message in SnapshotRevert.

In the error handling for c.EngineSnapshotRevert, the error message states "failed to create snapshot" instead of "failed to revert snapshot". Please correct the error message for accurate logging.

Apply this diff to fix the error message:

	if err != nil {
-		return nil, grpcstatus.Errorf(grpccodes.Internal, "failed to create snapshot %v: %v", req.Name, err)
+		return nil, grpcstatus.Errorf(grpccodes.Internal, "failed to revert snapshot %v: %v", req.Name, err)
	}

Line range hint 276-279: Implement the SnapshotClone method for V2 engines or handle unimplemented methods appropriately.

The SnapshotClone method in V2DataEngineProxyOps is currently unimplemented, returning an unimplemented error. This may lead to issues if the method is invoked for V2 engines. Consider implementing this functionality or providing proper error handling to prevent unexpected behavior.

Would you like assistance in implementing the SnapshotClone method for V2 engines? I can help draft the necessary code changes or open a GitHub issue to track this task.


Line range hint 315-318: Implement SnapshotHash and SnapshotHashStatus methods for V2 engines.

The methods SnapshotHash and SnapshotHashStatus in V2DataEngineProxyOps are currently unimplemented, returning unimplemented errors. To fully support the auto-salvage feature for V2 volumes, consider implementing these methods or ensuring they are appropriately handled to prevent runtime errors.

Would you like assistance in implementing these methods? I can help draft the code or open a GitHub issue to track this enhancement.

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between c60d290 and 227ad47.

⛔ Files ignored due to path filters (13)
  • go.mod is excluded by !go.mod
  • go.sum is excluded by !**/*.sum, !go.sum
  • vendor/github.com/longhorn/go-common-libs/utils/misc.go is excluded by !vendor/**
  • vendor/github.com/longhorn/longhorn-spdk-engine/pkg/client/client.go is excluded by !vendor/**
  • vendor/github.com/longhorn/longhorn-spdk-engine/pkg/spdk/engine.go is excluded by !vendor/**
  • vendor/github.com/longhorn/longhorn-spdk-engine/pkg/spdk/replica.go is excluded by !vendor/**
  • vendor/github.com/longhorn/longhorn-spdk-engine/pkg/spdk/server.go is excluded by !vendor/**
  • vendor/github.com/longhorn/types/pkg/generated/imrpc/instance.pb.go is excluded by !**/*.pb.go, !**/generated/**, !vendor/**
  • vendor/github.com/longhorn/types/pkg/generated/spdkrpc/spdk.pb.go is excluded by !**/*.pb.go, !**/generated/**, !vendor/**
  • vendor/golang.org/x/exp/LICENSE is excluded by !vendor/**
  • vendor/golang.org/x/exp/PATENTS is excluded by !vendor/**
  • vendor/golang.org/x/exp/constraints/constraints.go is excluded by !vendor/**
  • vendor/modules.txt is excluded by !vendor/**
📒 Files selected for processing (5)
  • pkg/client/instance.go (2 hunks)
  • pkg/instance/instance.go (2 hunks)
  • pkg/proxy/replica.go (2 hunks)
  • pkg/proxy/snapshot.go (1 hunks)
  • pkg/proxy/volume.go (1 hunks)
🔇 Additional comments (6)
pkg/proxy/volume.go (1)

Line range hint 141-143: Prioritize V2 method implementations based on auto salvage requirements.

Multiple V2 methods are marked as TODO. Given that this PR aims to support auto salvage for v2 volumes, it would be helpful to:

  1. Identify which methods are critical for the auto salvage feature
  2. Prioritize their implementation
  3. Document any dependencies between these methods

Let's check which methods might be required for auto salvage:

Also applies to: 182-184, 224-226, 266-268, 308-310, 350-352

pkg/client/instance.go (2)

94-94: LGTM: Clear and well-named field addition

The new SalvageRequested boolean field is appropriately named and aligns with the PR objective to support auto salvage functionality for v2 volumes.


151-151: Consider adding documentation for the salvage request functionality

While the implementation is correct, it would be beneficial to add documentation about:

  • When the SalvageRequested flag should be set
  • Any prerequisites or conditions that should be met before requesting salvage
  • Potential impacts of enabling salvage mode

Additionally, let's verify the integration with the SPDK layer:

pkg/proxy/snapshot.go (1)

138-138: Confirm the use of the boolean parameter in EngineGet.

In engine, err := c.EngineGet(req.EngineName, true), the second parameter true alters the behavior of EngineGet. Please ensure that this change is intentional and that it aligns with the function's expected parameters. Verify that passing true is appropriate in this context.

pkg/instance/instance.go (2)

154-154: Addition of 'SalvageRequested' parameter to EngineCreate

The inclusion of req.Spec.SpdkInstanceSpec.SalvageRequested in the EngineCreate call enables the engine to handle salvage requests for v2 data engines, supporting the auto salvage feature as intended.


273-273: Verify the use of 'false' in EngineGet and ReplicaGet calls

The addition of false as the second argument in c.EngineGet(req.Name, false) and c.ReplicaGet(req.Name, false) alters the behavior of instance retrieval. Please ensure that passing false is intentional and aligns with the expected functionality of these methods.

To confirm that false is the correct value, you can review the method signatures and documentation of EngineGet and ReplicaGet to ensure they behave as expected when this parameter is false.

Also applies to: 279-279

pkg/proxy/volume.go Outdated Show resolved Hide resolved
pkg/proxy/replica.go Outdated Show resolved Hide resolved
Copy link

mergify bot commented Oct 26, 2024

This pull request is now in conflict. Could you fix it @c3y1huang? 🙏

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Outside diff range and nitpick comments (1)
pkg/client/instance.go (1)

94-94: Add documentation for the new SalvageRequested field.

Please add a comment explaining the purpose and usage of this field to improve code maintainability.

 type EngineCreateRequest struct {
     ReplicaAddressMap map[string]string
     Frontend          string
     InitiatorAddress  string
     TargetAddress     string
     UpgradeRequired   bool
+    // SalvageRequested indicates whether a salvage operation should be performed during engine creation
     SalvageRequested  bool
 }
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between 227ad47 and f2ab2bf.

⛔ Files ignored due to path filters (13)
  • go.mod is excluded by !go.mod
  • go.sum is excluded by !**/*.sum, !go.sum
  • vendor/github.com/longhorn/go-common-libs/utils/misc.go is excluded by !vendor/**
  • vendor/github.com/longhorn/longhorn-spdk-engine/pkg/client/client.go is excluded by !vendor/**
  • vendor/github.com/longhorn/longhorn-spdk-engine/pkg/spdk/engine.go is excluded by !vendor/**
  • vendor/github.com/longhorn/longhorn-spdk-engine/pkg/spdk/replica.go is excluded by !vendor/**
  • vendor/github.com/longhorn/longhorn-spdk-engine/pkg/spdk/server.go is excluded by !vendor/**
  • vendor/github.com/longhorn/types/pkg/generated/imrpc/instance.pb.go is excluded by !**/*.pb.go, !**/generated/**, !vendor/**
  • vendor/github.com/longhorn/types/pkg/generated/spdkrpc/spdk.pb.go is excluded by !**/*.pb.go, !**/generated/**, !vendor/**
  • vendor/golang.org/x/exp/LICENSE is excluded by !vendor/**
  • vendor/golang.org/x/exp/PATENTS is excluded by !vendor/**
  • vendor/golang.org/x/exp/constraints/constraints.go is excluded by !vendor/**
  • vendor/modules.txt is excluded by !vendor/**
📒 Files selected for processing (2)
  • pkg/client/instance.go (2 hunks)
  • pkg/instance/instance.go (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • pkg/instance/instance.go
🔇 Additional comments (2)
pkg/client/instance.go (2)

Line range hint 94-151: Implementation looks good overall.

The changes appropriately implement the foundation for v2 volume auto-salvage support. The new field is well-integrated into the existing flow and maintains backward compatibility.


151-151: Verify RPC layer integration for SalvageRequested.

The field is correctly propagated to the SPDK instance spec. Let's verify that the RPC layer is prepared to handle this new field.

✅ Verification successful

Let me gather more information about the RPC implementation to verify the integration.


RPC layer integration for SalvageRequested is properly implemented

The field is correctly integrated across all required layers:

  • Defined in the protobuf spec (instance.pb.go and spdk.pb.go)
  • Properly propagated through the SPDK engine client and server implementations
  • Correctly structured in both EngineCreateRequest and SpdkInstanceSpec
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check if the RPC proto files have been updated to include the SalvageRequested field

# Search for SalvageRequested field definition in proto files
echo "Checking proto files for SalvageRequested field:"
rg -l "SalvageRequested" --type proto

# Check for any related RPC handler implementations
echo -e "\nChecking for RPC handler implementations:"
rg "SalvageRequested" --type go -g "**/*server*.go"

Length of output: 560


Script:

#!/bin/bash
# Check proto definitions in vendor directory
echo "Checking proto definitions in vendor directory:"
rg -l "SalvageRequested" --type=proto ./vendor

# Check the actual implementation in SPDK engine
echo -e "\nChecking SPDK engine implementation:"
rg "SalvageRequested" -A 5 ./vendor/github.com/longhorn/longhorn-spdk-engine/

# Check for any related struct definitions
echo -e "\nChecking related struct definitions:"
ast-grep --pattern 'type $_ struct {
  $$$
  SalvageRequested $_
  $$$
}'

Length of output: 6920

@c3y1huang c3y1huang force-pushed the 8430-feat-v2-auto-salvage branch 4 times, most recently from 2576ccf to 48cfc80 Compare November 1, 2024 01:20
go.mod Outdated Show resolved Hide resolved
@c3y1huang c3y1huang force-pushed the 8430-feat-v2-auto-salvage branch 2 times, most recently from f40b8f8 to 57e8bbb Compare November 1, 2024 08:28
Copy link

mergify bot commented Nov 3, 2024

This pull request is now in conflict. Could you fix it @c3y1huang? 🙏

Copy link
Member

@derekbit derekbit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Waiting for the merge of longhorn-spdk-engine's PR.

derekbit
derekbit previously approved these changes Nov 4, 2024
Copy link
Member

@derekbit derekbit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Waiting for the merge of longhorn-spdk-engine's PR.

@c3y1huang c3y1huang force-pushed the 8430-feat-v2-auto-salvage branch 2 times, most recently from 1d79722 to b3f1700 Compare November 6, 2024 02:49
longhorn/longhorn-8430

Signed-off-by: Chin-Ya Huang <chin-ya.huang@suse.com>
longhorn/longhorn-8430

Signed-off-by: Chin-Ya Huang <chin-ya.huang@suse.com>
@c3y1huang c3y1huang merged commit e580828 into longhorn:master Nov 6, 2024
12 checks passed
@c3y1huang c3y1huang deleted the 8430-feat-v2-auto-salvage branch November 6, 2024 04:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants